1. Field of the Invention
The present invention relates to a moving image encoding technique for real time variable bit rate encoding, and more particularly to a technique for guaranteeing recording time in advance when recording an encoded stream on a storage medium.
2. Description of the Related Art
Recent significant advances in digital signal processing technology have enabled recording of moving images on storage media and transfer of moving images through transmission lines, which were difficult to achieve in the past. For this, the respective pictures that constitute a moving image are compression-encoded to significantly reduce its data amount. As a typical method of this compression encoding process, there is, for example, the MPEG (Moving Picture Experts Group) scheme.
When a series of pictures are compression-encoded in accordance with the MPEG scheme with the condition of a constant bit rate, the encoded data amount differs significantly depending on a scene made up of a plurality of pictures, spatial frequency characteristics of pictures, the correlation between pictures, and the quantizer scale value. Encoded data amount control is an important technique for minimizing encoding distortion when achieving an apparatus that has such encoding characteristics.
Algorithms for realizing the encoded data amount control can be roughly classified into two types: constant bit rate encoding scheme (hereinafter referred to as “CBR scheme”) and variable bit rate encoding scheme (hereinafter referred to as “VBR scheme”). Ordinarily, the VBR scheme is known to provide decoded pictures of better image quality than the CBR scheme because the VBR scheme assigns codes adaptively depending on the difficulty level of encoding. The adaptive assignment of codes is realized by, for example, assigning a high bit rate to a scene with a high difficulty level of encoding and assigning a low bit rate to a scene with a low difficulty level of encoding.
As the CBR scheme, TM5 (Test Model 5 (Test Model Editing Committee: “Test Model 5”, ISO/IEC JTC/SC29/WG11/N0400 (April 1993))) that was proposed in the course of standardization of MPEG-2 encoding scheme and a scheme disclosed in Japanese Patent No. 3112035 are known.
Meanwhile, the following documents disclose a technique for achieving the VBR scheme in real time (i.e., 1-pass).    Japanese Patent Laid-Open No. H9-294267    Japanese Patent No. 3265818    Japanese Patent Laid-Open No. 2003-61051    Japanese Patent No. 3268306    Japanese Patent No. 3358620
Japanese Patent Laid-Open No. H9-294267 attempts to improve image quality with the VBR scheme by solving the problems encountered with the CBR scheme. In the CBR scheme, if the encoded data amount that is generated in a GOP (Group of Pictures) is excessively greater than the target encoded data amount of the GOP, the next target encoded data amount tries to absorb this surplus, resulting in the problem of image quality degradation. FIG. 2 shows a transition in encoded data amount generated on a GOP-by-GOP basis according to the VBR scheme of Japanese Patent Laid-Open No. H9-294267. It can be seen from FIG. 2 that the generated encoded data amount increases sharply in GOP5 because GOP5 has a high difficulty level of encoding. For this reason, the generated encoded data amount decreases gradually from GOP6 toward GOP9. In order to achieve such a gradual change in encoded data amount generated in GOPs, the target encoded data amount R(j) of the jth GOP is determined using the following equations.DIFF(j)=DIFF(j−1)+B(j−1)−Tg  (1)R(j)=Tg−(1/T)×DIFF(j)  (2)
In the equations, B(j) represents an encoded data amount generated in the jth GOP, Tg represents a target encoded data amount equally allocated to each GOP based on a given target bit rate, and T is a constant equal to or greater than 2. As is obvious from the equations (1) and (2), the target encoded data amount R(j) is determined according to DIFF(j), which is the accumulation of the encoded data amounts generated in GOPs in the past.
According to Japanese Patent No. 3265818 and Japanese Patent Laid-Open No. 2003-61051, a feedforward type VBR scheme is achieved by performing detection on a picture group made up of a plurality of pictures and a picture to be encoded using a means for detecting difficulty level of encoding, called “encoding difficulty information calculation units 301 and 302” as shown in FIG. 3. According to this method, a picture group made up of a plurality of pictures is divided by a picture group dividing unit 300, and the difficulty level of encoding of the picture group is calculated by the encoding difficulty information calculation unit 301. Because target encoded data amount calculation units 303 and 304 can variably assign a target encoded data amount to the picture group according to the calculated difficulty level of encoding, a variation in image quality of decoded pictures can be suppressed.
However, the above-described conventional techniques have the following problems.
First, in the case of Japanese Patent Laid-Open No. H9-294267, when determining a target encoded data amount R(j) for a GOP to be encoded, the target encoded data amount is not determined as appropriate according to the difficulty level of encoding because reference is made only to the encoded data amounts generated for the already encoded GOPs (the zeroth, first, . . . , the (j−1)th GOP). In the case of Japanese Patent Laid-Open No. H9-294267, focus is given on the target encoded data amounts of GOPs after the occurrence of the sharp increase in the generated encoded data amount, and merely the encoded data amount is caused to decrease gradually. The image quality degradation of decoded pictures are improved locally as compared to the CBR scheme, but it is difficult to improve the degradation of the image quality of decoded pictures throughout a sequence.
In the case of Japanese Patent No. 3265818 and Japanese Patent Laid-Open No. 2003-61051, the average and maximum bit rates are guaranteed, but the maximum encoded data amount is not guaranteed. This means, when recording on a storage medium of an imaging device such as a digital video camera, the user may not know in advance how long the recording takes. In this case, even if the user specifies a bit rate before the start of imaging, a situation may occur in which the generated encoded data amount of a sequence exceeds the product obtained by multiplying the bit rate by the imaging time.
In the case of Japanese Patent Laid-Open No. 2003-61051, the encoding difficulty information calculation units 301 and 302 require an encoding means similar to an encoding unit 305, which makes the processing load very heavy. This Japanese Patent Laid-Open No. 2003-61051 discloses the use of spatial activity to determine a difficulty level of encoding, but the use of spatial activity is insufficient to predict a difficulty level of encoding in the encoding unit 305.
There is another method, called 2-pass encoding method, as disclosed in Japanese Patents Nos. 3268306 and 3358620, which disclose a method for achieving a VBR scheme. According to this method, a difficulty level of encoding of each scene is extracted during the first pass, and in the second pass, a target encoded data amount is assigned to each scene according to the extracted difficulty level of encoding. Accordingly, the image quality can be improved throughout a sequence as compared to the CBR scheme. Imaging devices, however, require real time encoding, and therefore it is difficult to implement 2-pass encoding.